Comments for MEDB 5501, Week 8

Review multiple comparisons issue

  • Type I error: rejecting the null hypothesis when the null hypothesis is true.
    • Multiple simultaneous hypotheses increase the Type I error rate.

Bonferroni inequality for two simultaneous hypotheses

Define

  • \(E_1\) = Type I error for Hypothesis 1
  • \(E_2\) = Type I error for Hypothesis 2
    • \(P[E_1\ \cup\ E_2]=P[E_1]+P[E_2]-P[E_1\ \cap\ E_2]\)
    • \(P[E_1\ \cup\ E_2]\ \le\ P[E_1]+P[E_2]\)
    • \(P[E_1\ \cup\ E_2]\ \le\ P[E_1]+P[E_2]\)
    • \(P[E_1\ \cup\ E_2]\ \le\ 2 \alpha\)

Bonferroni adjustment

  • For m hypotheses
    • \(P[E_1\ \cup\ ...\ \cup E_m]\ \le\ m \alpha\)
  • Test each hypothesis at \(\alpha/m\)
    • Preserves overall Type I error rate
  • Example, 3 simultaneous hypotheses
    • Reject H0 if p-value < 0.0133

Controversies over Bonferroni adjustment

  • Increases Type II errors
  • Impractical for large values of m
  • Works poorly for highly correlated tests
  • Ambiguity in definition of “simultaneous” hypotheses

Alternatives to Bonferroni adjustment

  • False discovery rate
  • Designation of primary and secondary outcomes
  • Subjective assessment of simultaneous hypotheses

Review two-sample t-test

  • \(H_0:\ \mu_1=\mu_2\)
  • \(H_1:\ \mu_1 \ne \mu_2\)
    • Or a one tailed alternative
  • \(T=\frac{\bar X_1-\bar X_2}{SE(\bar X_1-\bar X_2)}\)
    • Accept H0 if T is close to zero.

What to do with three or more groups?

  • \(H_0:\ \mu_1=\mu_2=...=\mu_k\)
  • \(H_1:\ \mu_i \ne \mu_j\) for some i, j
    • Note: one-tailed test is tricky.
  • Accept H0 if the F ratio (defined below) is close to 1.

Important assumptions

  • Same as independent-samples t-test
    • Normality
    • Equal variances
    • Independence

How to check assumptions

  • Boxplots
  • Analysis of residuals, \(e_{ij}\)
    • \(e_{ij}\)= Observed - Predicted
    • \(e_{ij}= Y_{ij}-\bar{Y}_i\)

Tukey post hoc tests

  • If you reject H0, which values are unequal
    • With k groups, there are k(k-1)/2 comparisons
  • Studentized range (Tukey test)
    • Requires equal sample sizes per group
    • Uses harmonic mean approximation for unequal sample sizes.
      • Do not use harmonic means if seriously different sample sizes.

Alternatives to Tukey post hoc tests

  • Bonferroni adjustment
    • Works for unequal sample sizes per group
    • Works for unequal variances
  • Dunnett’s test
    • Treatment versus multiple controls
  • Scheffe’s test
    • Works for complex comparison
      • Example \(\mu_1\ vs.\ \frac{\mu_2+\mu_3+\mu_4}{3}\)

Artificial data

   g  y
1  1 23
2  1 30
3  1 25
4  2 33
5  2 36
6  2 41
7  2 37
8  2 43
9  3 24
10 3 29
11 3 31

Scatterplot

Total SS = 452

Within SS = 116

Between SS = 336

Degrees of freedom

  • For Total SS, df = 10
  • For Within SS, df = 8
  • For Between SS, df = 2
  • In general,
    • N = number of observations
    • k = number of groups
      • Total df = N-1
      • Within df = N-k
      • Between df = k-1

ANOVA table, 1 of 4

ANOVA table, 2 of 4

ANOVA table, 3 of 4

ANOVA table, 4 of 4

ANOVA table from the general linear model

Regression approach to ANOVA, 1 of 4

Regression approach to ANOVA, 2 of 4

Regression approach to ANOVA, 3 of 4

Regression approach to ANOVA, 4 of 4

Predicted values and residuals, 1 of 2

Predicted values and residuals, 2 of 2

Recode into different variables dialog box

Old and new Values dialog box

General Linear Model | Univariate dialog box

Descriptives output

Boxplot

Analysis of variance table

Analysis of variance table with irrelevant rows removed

Parameter estimates

Tukey post hoc test, 1 of 7

Tukey post hoc test, 2 of 7

Tukey post hoc test, 3 of 7

Tukey post hoc test, 4 of 7

Tukey post hoc test, 5 of 7

Tukey post hoc test, 6 of 7

Tukey post hoc test, 7 of 7

Checking assumptions, boxplots

Checking assumptions, descriptive statistics

Residual analysis, 1 of 2

Residual analysis, 2 of 2

Analysis of school year, 1 of 4

Analysis of school year, 2 of 4

Analysis of school year, 3 of 4

Analysis of school year, 4 of 4